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ABSTRACT 

Enhanced Vision (EV) and synthetic vision (SV) systems may serve as enabling technologies to meet the challenges of 
the Next Generation Air Transportation System (NextGen) Equivalent Visual Operations (EVO) concept - that is, the 
ability to achieve or even improve on the safety of Visual Flight Rules (VFR) operations, maintain the operational 
tempos of VFR, and even, perhaps, retain VFR procedures independent of actual weather and visibility conditions. One 
significant challenge lies in the definition of required equipage on the aircraft and on the airport to enable the EVO 
concept objective. A piloted simulation experiment was conducted to evaluate the effects of the presence or absence of 
Synthetic Vision, the location of this information during an instrument approach (i.e., on a Head-Up or Head-Down 
Primary Flight Display), and the type of airport lighting information on landing minima. The quantitative data from this 
experiment were analyzed to begin the definition of performance-based criteria for all-weather approach and landing 
operations. Objective results from the present study showed that better approach performance was attainable with the 
head-up display (HUD) compared to the head-down display (HDD). A slight performance improvement in HDD 
performance was shown when S V was added, as the pilots descended below 200 ft to a 1 00 ft decision altitude, but this 
performance was not tested for statistical significance (nor was it expected to be statistically significant). The 
touchdown data showed that regardless of the display concept flown (SV HUD, Baseline HUD, SV HDD, Baseline 
HDD) a majority of the runs were within the performance-based defined approach and landing criteria in all the visibility 
levels, approach lighting systems, and decision altitudes tested. For this visual flight maneuver, RVR appeared to be the 
most significant influence in touchdown performance. The approach lighting system clearly impacted the pilot’s ability 
to descend to 100 ft height above touchdown based on existing Federal Aviation Regulation (FAR) 91.175 using a 200 ft 
decision height, but did not appear to influence touchdown performance or approach path maintenance. 

Keywords: Synthetic Vision System, Head-Up Display, Global Positioning System, Aviation Safety, Next Generation 
Air Transportation System, Approach Fighting System, Equivalent Visual Operations 

1. INTRODUCTION 

The Integrated Intelligent Flight Deck Technologies (IIFDT) project, under NASA’s Aviation Safety Program (AvSP), 
comprises a multi-disciplinary research effort to develop flight deck technologies that mitigate operator-, automation-, 
and environment-induced hazards. Towards this objective, IIFDT is developing crew/vehicle interface technologies that 
reduce the propensity for pilot error, minimize the risks associated with pilot error, and proactively overcome aircraft 
safety barriers that would otherwise constrain the full realization of the next generation air transportation system 
(NextGen). 1 Part of this research effort involves the use of enhanced and synthetic vision systems and other interface 
modalities as enabling technologies to meet the challenges of an Equivalent Visual Operations (EVO) concept - that is, 
the ability to achieve or even improve on the safety of Visual Flight Rules (VFR) operations, maintain the operational 
tempos of VFR, and even, perhaps, retain VFR procedures independent of actual weather and visibility conditions. 

One significant challenge to the EVO concept objective is the definition of required equipage on the aircraft and on the 
airport. With today’s equipment and regulations, significant investment is required in on-board equipment for 
navigation, surveillance, and flight control and on the airport for precision guidance systems and approach lighting 
systems for “all-weather” landing capability. The levels of equipment redundancy, capability, and accuracy dramatically 
increase as landing visibility minima decrease. The necessity for this equipment and the applicability of these 
regulations given new technologies such as enhanced vision (EV) and synthetic vision (SV) should be reevaluated. 
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A vivid demonstration of the current complexity (and cost) to meet “weather and visibility-independent” capability is to 
look at the present National Air Space (NAS) infrastructure. As of January 2008, there were 1,229 ILS instrument 
approach procedures (IAPs) to Category I minima (no lower than 200 ft decision height, 2400 ft visibility) available 
throughout the United States, but only 143 Category II (no lower than 100 ft decision height, 1200 ft visibility) and 111 
Category III (a decision height lower than 100 ft, or no decision height, or a runway visual range (RVR) less than 1200 
ft) IAPs 2 using 1,367 Instrument Landing Systems (ILS) in the NAS. 3 In addition to precision guidance ILS, approach 
lighting systems of increasing complexity are required by regulation as landing visibility minima decrease. Typical for 
Category I precision approaches is the MALSR (Medium Intensity Approach Lighting System with Runway Alignment 
Indicator Lights) approach lighting system (ALS). Currently, there are approximately 900 MALSR in the NAS. 4 For 
Category II/III landing minima, the ALSF-2 (High Intensity Approach Lighting System with Sequenced Flashing Lights) 
ALS is installed to provide the visual information for runway alignment, height perception, roll guidance, and horizontal 
references. In Figure 1, a comparative view of different ALSs is shown, including a typical “VFR” (Visual Flight Rules) 
configuration (which would not support an instrument approach procedure under the current regulations). By FAA 
estimates 5 , the facilities and equipment (F&E) costs for a single MALSR installation was nearly $1M, representing 
almost 50% of the total F&E costs for a Category I capability. The ALSF-2 costs are nearly 3 times this amount and 
they comprise over 70% of the total F&E costs for a Category II/III capability. The 20 year life cycle costs for the 
MALSR were estimated at $1.7M, climbing to $5.4M for a ALSF-2 light system with a Category III landing minima. 



Figure 1. Approach Lighting System Configurations - VFR (left), MALSR (center), and ALSF-2 (right) 


To meet the operational goals of NextGen, all-weather, Category Ill-type capability is desired for all runways. This goal 
is economically unlikely, however, if these infrastructure requirements remain in place. Technology innovation may 
provide an avenue by which the safety and performance provided by this infrastructure can be economically replaced. 
Synthetic Vision and Enhanced Vision capability may offer crew/vehicle interface technologies that help achieve this 
goal and the realization of EVO in the next generation NAS by obviating the requirement for elaborate ALSs and other 
infrastructure needs to support lower landing minima. 


2. BACKGROUND 

Synthetic vision is a computer-generated image of the external scene topography that is generated from aircraft attitude, 
high-precision navigation, and data of the terrain, obstacles, cultural features, and other required flight information. A 
synthetic vision system (SVS) enhances this basic functionality with real-time integrity to ensure the validity of the 
databases, perform obstacle detection and independent navigation accuracy verification, and provide traffic surveillance. 
Under NASA’s Aviation Safety Program/Synthetic Vision Project (1999-2006), NASA and its industry partners 






developed and deployed SVS technologies for commercial and business aircraft which were shown to provide significant 
improvements in terrain awareness and reductions for the potential of Controlled-Flight-Into-Terrain incidents/accidents 
compared to current generation cockpit technologies. 6 " 11 

It has been hypothesized that the use of SV technologies on head-up and head-down displays can provide precision 
approach, landing, and taxi guidance for “all weather” capability to all runways without requiring extensive approach 
lighting systems, ground-based precision guidance systems such as the ILS, or other airport infrastructure. These 
technologies may provide “equivalent vision” capabilities which would obviate the need for airport lighting and other 
infrastructure to support the flight crews’ need to visually acquire runways and taxiways using normal vision. 

An experiment was conducted to evaluate the effects of the presence or absence of Synthetic Vision, the location of this 
information during an instrument approach (i.e., on a Head-Up or Head-Down Primary Flight Display), and the type of 
airport lighting information on landing minima. The “operational considerations” evaluated under this effort included 
reduced visibility, decision altitudes, and airport approach lighting systems, for SVS-equipped and non-equipped 
aircraft. Another key element of the testing entailed investigating the pilot’s awareness and reaction to non-normal 
events (i.e., failure conditions) that were unexpectedly introduced into the experiment. These non-normals are critical 
determinants in the underlying safety of all-weather operations. 

The main objectives of this experiment were to: 

1) Evaluate SVS and SVS-related technologies to develop operational concepts for all-weather approach and 
landing; and, 

2) Provide quantitative and qualitative information that could be used to develop criteria for all-weather 
approach and landing technologies such as synthetic vision. 

The qualitative data and experimental hypotheses, focusing on the Objective 1 noted above, have been reported in detail 
in Reference 12. 

In this paper, the experimental data are evaluated using proposed performance-based approach and landing standards in 
addressing Objective 2 noted above. The results are presented within the context of emerging operational concepts for 
EVO. In addition, the effects of reduced visibility, decision altitudes, and airport equipage requirements on SVS- 
equipped and non-equipped aircraft, the role of display location (Head-Down Display (HDD) vs. Head-Up Display 
(HUD)) in landing operations and the pilot decision-making process are also investigated. 

3. EXPERIMENT 

In the following section, a brief review of the experiment is given as background for the analyses that follow in Section 
4. Additional details for the experiment are contained in Reference 12. 

3.1. Simulation 

The experiment was conducted in the Integration Flight Deck (IFD) full-mission simulator facility (see Fig. 2) at NASA 
Langley Research Center (LaRC). A collimated out-the-window (OTW) scene provided 200 degrees horizontal by 40 
degrees vertical field-of-view at 26 pixels per degree. 



Figure 2. Integration Flight Deck Simulation Facility with HUD and Head-Down Research Display (HD-RD). 


The evaluation pilot (EP) occupied the left seat, as the Pilot Flying (PF) for this experiment. The left seat included an 
overhead HUD projection unit and a head-down research display (RD) (see Fig. 2). Twenty-three pilots, representing 
seven airlines and the Federal Aviation Administration (FAA) Flight Standards and Transport Aircraft Certification 
Branches, participated in the experiment. 


3.2. Displays 

The HUD subtended approximately 26° horizontal by 21° vertical field of view (FOV). The input consisted of a video 
mix of symbology and computer-generated scene imagery. The symbology included “haloing” to ensure that the 
symbology was highlighted against the scene imagery background. HUD brightness, contrast and “declutter” controls 
were provided. The SV imagery, when displayed, was drawn conformally. 

A head-down research display was installed over the normal instruments on the left hand side of the IFD cockpit (see Fig 
22). Two Size D (6.4 x 6.4 inch) primary flight display (PFD) and navigation displays (ND) were drawn. 

3.3. Synthetic Vision Database 

A synthetic vision database was created from a 0.33 arc-sec (~10 meter post-spacing) Digital Elevation Model (DEM) of 
a 53 x 57 nm area centered around the Dallas- Ft. Worth International Airport (FAA identifier: KDFW). The DEM was 
draped with elevation-based generic imagery. The airport was represented by three-dimension models of the runway, 
taxiways, and terminal buildings which were extracted from aerial photography of approximately 1 foot resolution. 

3.4. Symbology 

3.4.1. HUD Symbology 

The HUD stroke symbology is shown in Figure 3. Glideslope and localizer raw data “path deviation” indicators were 
provided in addition to Flight Path Marker (FPM) and guidance information. The pitch-roll guidance cue (“ball”) used 
modified pursuit guidance 13 along the desired path centerline, 5.5 seconds ahead of ownship. Horizontal and vertical 
position of the ball reflects the track and flight path angles to fly to the center of the desired path. A glideslope reference 
line was drawn at the DFW Runway 18R Instrument Fanding System (ITS) descent angle of 3.0 degrees. Also, a 
runway outline symbol was drawn using the threshold coordinates of the DFW 18R/36F runway based on the simulated 
aircraft navigation solution to conformally position the symbol. The runway outline was drawn using an 8000 ft x 200 ft 
runway. Finally, radar altitude was shown digitally underneath the FPM when below 500 ft above ground level (AGE). 



Figure 3. HUD Symbology 


3.4.2. PFD Symbology 

The PFD symbology was essentially the same as the HUD. See Reference 12 for details. 

3.5. Display Concepts 

Four display concepts (baseline and SV, either head-up or head-down) were evaluated by the evaluation pilots while 
flying approaches to DFW Runway 1 8R. The head-down navigation display format was invariant. 

3.5.1. Head-Up Display Concepts 

Two HUD display concepts were tested, differing from each other only in the type of raster background (SV or none) 
presented. Standard HUD symbology enhanced with a runway outline was employed in both HUD concepts. In Figure 
4, the two HUD Concepts are shown - the Baseline HUD (left) and the SV HUD (right). During the HUD experimental 
runs, the “baseline” PFD was displayed (PFD format shown on the left side of Figure 5). 
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Figure 4. Head-Up Display (HUD) Formats - Baseline-HUD (left) and SV-HUD (right). 


3.5.2. Head-Down Display Concepts 

Two PFD display concepts were tested, differing from each other only in the type of background (standard sky/ground or 
SV) presented with the flight symbology. Figure 5 presents the PFD concepts used for the Baseline HDD (left) and the 
Synthetic Vision HDD (right) configurations. For the HDD evaluations, the HUD was stowed to preclude blocking or 
distortion of the pilot’s forward view of the outside world. 







3.6. Approach Lighting System Configurations 

Three different ALS configurations were used for this experiment as shown in Figure 1. The first ALS configuration 
(hereinafter referred to as the VFR ALS) was representative of lights typically found at a VFR runway and included 
Runway End Identification Lights (REIL), precision approach path indicator lights (PAPI), partial threshold lights, and 
medium intensity runway lights (MIRL). The second ALS configuration (hereinafter referred to as the MALSR ALS) 
was representative of a Category I/Type I operations runway and included MALSR (medium intensity approach lighting 
system; runway alignment indicator lights), REIL, PAPI, full threshold lights, and MIRL. The third ALS configuration 
(hereinafter referred to as the ALSF-2 ALS) was representative of a standard Category II/Type II operations runway and 
included ALSF-2 (high intensity approach lighting system with sequenced flashing lights), touchdown zone and 
centerline lighting, REIL, PAPI, full threshold lights, and high intensity runway lights (FURL). 

3.7. Evaluation Task 

The evaluation task was a straight-in Global Positioning System (GPS) approach with a 3 degree descent angle to 
Runway 18R at DFW airport. For each run, the approach started 5 nautical miles (nm) from the runway threshold and 
the aircraft was placed one dot left or right of the course centerline, and one dot high or low of the course glidepath. The 
weather consisted of altitude-based cross winds (wind direction and intensity was dependent on altitude), light 
turbulence (root-mean-square (rms) of 2 ft/sec), and varying visibility levels (3 miles, 2400 ft, 1800 ft, or 1200 ft RVR). 
The evaluation pilot hand- flew the approach from the left seat with auto-throttles engaged at a speed of 138 knots. The 
run was terminated at full-stop or upon go-around initiation. The aircraft was configured to land prior to each run 
(landing gear down and flaps 30 degrees). There was no other aircraft or ATC involvement in the task. 

The evaluation pilots were instructed to fly the aircraft as if there were passengers aboard, fly the center of the approach 
path, and land as close as possible to the centerline and touchdown zone (1000 feet from the runway threshold). They 
were also instructed to initiate a go-around if the landing was not safe or if there were any safety concerns. 

Each approach used a decision altitude (DA) of either 200 ft or 100 ft. The DA was briefed before each run. 

An automatic aural call-out was included in the simulation to “assist” in altitude awareness. A “500 feet” call-out was 
made at 500 ft above field level (AFL). An “Approaching Minimums” call-out was made at 100 ft above the DA for the 
run (100 or 200 ft AFL). Finally, a “Minimums” call-out was made at the pre-set DA. 

The EPs were instructed to verbally denote and acknowledge their recognition of the required runway visual references 
in accordance with existing FAR 91.175 regulations. The pilot was instructed to call “Lights” when the approach 
lighting system became visible and to call “Landing” when the required landing visual references (as specified in FAR 
Part 91.175c) became identifiable. The procedures were as follows: 

• For approaches with a DA of 200 ft, the pilot had to visually acquire the approach lighting system or the 
runway landing references no later than the “minimums” call-out to continue the approach below 200 ft AFL. 
If the approach continued, the pilot continued to landing only if the required landing visual references became 
identifiable before 100 ft AFL, else a call of “go-around” was required. 

• For approaches with a DA of 100 ft, there was an automatic call-out of “Approaching Minimums” at 200 ft 
AFL and an automatic call-out of “Minimums” at 100 ft AFL. If the “Landing” call was made before 100 ft 
AFL, the approach could continue to landing, else a call of “go-around” was required. 

3.8. Experiment Matrix 

Nominally, ten training runs and fifty- three experimental runs were completed by each EP. 

The primary experiment matrix consisted of combinations (but not a full-factorial) of Display Concept (Baseline-HDD, 
SV-HDD, Baseline-HUD, or SV-HUD), ALS (VFR, MALSR, or ALSF-2), runway visibility range (1200 ft, 1800 ft, 
2400 ft, or 3 statute miles), and DA (100 or 200 ft) as shown in Table 1. Two additional runs (one at 2400 ft RVR/ 200 
ft DA/ MALSR ALS and one at 3 mile RVR/ VFR ALS) using the Baseline HDD concept but without flight director- 
type guidance were also completed by each EP. 

A significant component of the test, in addition to the nominal runs, was the investigation of the ability of the EP to 
recognize and properly handle non-normal events. Seven non-normal runs were flown by each EP. The non-normal 
runs were four database integrity monitoring scenarios, two altimetry failure scenarios, and one SV obstacle placement 
error scenario. The non-normal results and their implications of safety are discussed in Reference 12. 



Table 1: Primary Experiment Matrix 


Visibility Decision Altitude 

VFR ALS 

MALSR 

ALSF-2 

1200 ft RVR 100 ft 

V 

V 

v 

200 ft 


V 


1800 ft RVR 100 ft 


V 


200 ft 


V 


2400 ft RVR 100 ft 

v 

V 

V 

200 ft 


V 


3 statute miles n/a 

v 




Note: the checked cases were evaluated using each of the four display concepts: 
Baseline PFD, SV PFD, Baseline HUD, and SV HUD 


3.9. Test Conduct 

The subjects were given a 1-hour briefing, explaining the HUD and PFD concepts, pilot procedures, and the evaluation 
tasks. After the briefing, a 1-hour training session in the IFD was conducted to familiarize the subjects with the aircraft 
handling qualities, display symbologies, pilot procedures, and controls. The ‘rare-evenf (or non-normal) scenarios were 
not discussed, although the pilot’s responsibility for maintaining safe operations at all times was stressed. 

4. RESULTS 

In the following section, the experimental data analyses are presented. Existing performance-based approach and 
landing criteria associated with related technology are assessed for applicability as criteria for emerging technologies to 
meet all-weather approach and landing operations. These analyses additionally address the effects of reduced visibility, 
decision altitudes, and airport equipage with SVS-equipped and non-equipped aircraft and the role of display location 
(HDD vs HUD) in approach and landing operations. 

For the objective performance measures, analysis of variance (ANOVA) was conducted for the factors of display 
location (HDD, HUD), display information content (Baseline, SV), ALS (VFR, MALSR, ALSF-2), RVR (1200, 1800, 
2400), and DA (100, 200) where appropriate. When necessary, Student-Newman-Keuls (SNK) post-hoc tests with a set 
at 0.05 were performed. 

4.1. Experimental Measures 

During each run, aircraft state data (i.e., path error, pilot control inputs, sink rate at touchdown and touchdown location) 
were recorded for later analysis. After each run, pilots completed a run questionnaire consisting of the NASA Task Load 
Index (TLX) workload rating, 14 Situation Awareness Rating Technique (S ART), 14 and six Likert- type (5-point) 
questions specific to different constructs of making a stabilized and safe approach. 15 After data collection was 
completed, pilots were administered the Situation Awareness - Subjective Workload Dominance (SA- SWORD) 16 and 
Subjective Workload Dominance (SWORD) 14 tests for the display concept (Baseline-HDD, SV-HDD, Baseline-HUD, 
SV-HUD) comparisons. The pilots also completed a post-test questionnaire designed to elicit comments on the display 
concepts with the various approach lighting systems, the visual transition strategy employed by the pilot with the 
different display concepts, the pilot procedures, and the fidelity of the simulator. 

4.2. Subjective Results 

The subjective data analyses were previously reported 12 and are summarized below: 

• Post-test paired comparison tests for situation awareness (SA) and pilot workload showed that the SV-HUD 
provided the highest SA and lowest workload for any of the tested display concepts. The lowest SA and 
highest workload was associated with the Baseline-HDD concept. The pilot workload associated with the SV- 
HDD and the Baseline-HUD was rated by the pilots as being equal but the SA provided by the SV-PFD was 
rated significantly better than the Baseline-HUD. 




• Post-run subjective ratings included the effects of RVR and ALS, as well as the display concepts: 

• The TLX data showed that the addition of SV on the HUD provided significant pilot workload 
reduction, but it did not significantly improve pilot workload when added to the HDD. The workload in 
flying the 1200 ft RVR approaches was significantly higher than the 2400 ft and 1800 ft RVR approaches 
(which were not significantly different from each other). The type of ALS employed did not affect pilot 
workload. 

• The SART data showed that the addition of SV on both the HUD and HDD provided significant SA 
improvements, and the HUD provided significantly improved SA over the HDD. SA while flying the 1200 
ft RVR approaches was significantly lower than the 2400 ft and 1800 ft RVR approaches (which were not 
significantly different from each other). The type of ALS employed did not affect SA. 

• Post-run ratings indicated that the pilots felt that their lateral alignment and roll guidance were significantly 
improved when using SV compared to the baseline condition and when using the HUD versus the HDD, but no 
significant differences were found among the approach lighting systems or visibility conditions. In addition, 
there were no operationally significant differences for the approach lighting systems or visibility conditions in 
pilot post-run ratings of their ability to find the runway and touchdown zone references or to identify the 
necessary visual references when transitioning from instrument to visual flight to safely execute the landing. In 
fact, even with RVR as low as 1200 ft, the pilot’s felt, on average, that they could safely complete the approach 
and landing with the display configurations and lighting conditions tested. 

These data showed that the ALS had no effect on workload or situation awareness during low visibility approaches. 
However, the one area where the ALS did have a profound effect was on the ability to continue the approach below the 
DA; that is, the pilot’s ability to see the required landing references. As shown in Figure 6, the percentage of approaches 
flown to touchdown reduced as the RVR decreased, to the extent that an approach flown on a VFR ALS in 1200 ft RVR 
had only a 50-50 chance of concluding to a touchdown, even when the pilots flew to a 100 ft DA. For this same 
condition, with either a MALSR or ALSF-2 ALS, the data shows a significant improvement in completed approaches 
(now 85-88%). With a 100 ft decision height, the number of completed approaches is independent of MALSR and 
ALSF-2. If a 100 ft reduction in DA (from 200 ft to 100 ft) is possible, the data shows a 20% improvement in the 
percentage of completed landings when using the MALSR, in 1800 and 1200 ft RVR. 



Figure 6. Percentage of Completed Landings by RVR and ALS 
4.3. Objective Approach Performance 

Approach performance was first assessed using rms localizer and rms glide slope deviation (in dots); maximum bank 
angle (deg); and rms sink rate (feet per minute, fpm). These parameters correspond intuitively to the establishment and 
maintenance of a stabilized approach to landing - an important safety measure 17,20 . 


The approach data were analyzed from 900 ft to DA (either 200 or 100 ft). The 900 ft altitude was arbitrarily applied to 
reduce the potential bias introduced by the experiment set-up, in which all approaches were initiated with an intentional 
lateral and vertical offset. Guidance cueing would be expected to lead the aircraft to the path intercept at 900 ft AGL. 

4.3.1. Path Deviation 

Localizer angular deviation: The main factor of location (F(l,56)=14.43, p<0.001) and the second order interactions of 
RVR x DA (F(2,591)=9.53, p<0.001), location x RVR (F(2,591)=6.61, p=0.001), lights x location (F(2,591)=6.59, 
p=0.001), lights x content (F(2,591)=3.08, p=0.047), DA x location (F(l, 591)=4.40, p=0.036), and location x content 
(F(l,591)=4.28, p=0.039) were significant for rms localizer deviation. The other main factors (content, RVR, ALS, and 
DA) and remaining second order interactions were not significant (p>0.05) for this measure. Pilots had less rms 
localizer deviation when flying: 1) the HUD (mean=0.060 dots and standard deviation, g= 0.036) versus the HDD 
(mean=0.068 dots and g= 0.029). The interaction effects in this case were not operationally significant and are not 
reported. 

Glide slope angular deviation: The main factors of DA (F(l, 24)=59.20, p<0.001) and location (F(l,47)=15.64, 

p<0.001) and the second order interaction of DA x location (F(l,591)=3.89, p=0.049) were significant for rms glide 
slope deviation. The other main factors (content, RVR, and ALS) and second order interactions were not significant 
(p>0.05) for this measure. Pilots flew more precise glide path: 1) to a 200 ft DA (mean=0.262 dots and a=0.147) than to 
a 100 ft DA (mean=0.338 dots and g= 0.177) and 2) with the HUD (mean=0.270 dots and g= 0.141) compared to the 
HDD (mean=0.366 dots and a=0.188). 

4.3.2. Roll Stability 

The main factors of location (F(l,46)=16.94, p<0.001) and RVR (F(2,83)=3.77, p=0.027) and the second order 
interaction of location x content (F(l, 591)=4.65, p=0.031) were significant for maximum bank angle. The other main 
factors and all the second order interactions were not significant (p>0.05) for this measure. Greater roll stability (less 
maximum bank angle) was achieved when flying the HUD (mean=5.1 deg) vs. the HDD (mean=6.0 deg). Post-hoc tests 
(SNK using a=0.05) showed three unique subsets for bank angle with the 3 RVR values: 1) 2400 ft (mean=5.2 deg), 2) 
1800 ft (mean=5.6 deg) and 3) 1200 ft (mean=5.9 deg) - these differences are operationally inconsequential. Visual 
inspection of the data revealed that all runs were flown with bank angles of less than 30 degrees from 1000 ft AGL to 
touchdown which is what is expected from professional pilots during a stabilized approach. 17 

4.3.3. Descent Stability 

The main factor of location (F(l,31)=31.96, p<0.001) and the second order interactions of DA x location 
(F(l,591)=4.67, p=0.031), DA x content (F(l,591)=4.67, p=0.031) and location x content (F(l, 591)= 9.53, p=0.002) 
were significant for rms sink rate deviation. None of the other main factors or second order interactions were significant 
(p>0.05) for this measure. Greater descent stability (lower rms sink rate deviation from nominal value of 723 fpm) was 
achieved when flying the HUD (mean=l 19 fpm and a=38) versus the HDD (mean=165 fpm and g= 58). The interaction 
effects in this case were not operationally significant and are not reported. 

4.4. Objective Approach Standards Application 

Existing performance-based approach standards were also applied in the objective data analysis. These standards were 
drawn from numerous sources pertaining to the general concept of low- visibility approach and landings. However, none 
of these were written specifically as quantitative performance standards for Synthetic and Enhanced System operations. 
The analyses that follow examine their applicability for this purpose. (The notable exception is that the Enhanced Flight 
Vision Systems Advisory Circular (AC No. 90-EFVS (Draft)), states that “the use of EFVS must achieve the same 
degree of accuracy and flight technical error as a Category II integrated instrument landing system approach.”) 

A synopsis of these existing quantitative performance requirements for glideslope and localizer tracking from FAA and 
Joint Aviation Regulations (JAR) are shown in Table II. In Reference 19, glideslope and localizer performance 
requirements are expressed in microamps, with 150 microamps equal to full scale deflection on the ILS. In Table II, 
glideslope and localizer deviations are expressed in dots deflection by assuming +/- 2 dots full scale deflection 
corresponds to +/-150 microamps deviation from on-course. 

This synopsis emphasizes “performance” parameters of interest that are relevant to this experiment and does not include 
many important regulation facets and nuances for the sake of brevity. Of particular note, airspeed, sink rate, and bank 



angle control are typically evaluated but are not reported herein since autothrottles were engaged for all runs and sink 
rate and bank angle control differences among the main factors were found to be operationally inconsequential as 
described in Section 4.3 above. 

The existing quantitative performance standards emphasize the maximum glideslope and localizer deviations, instead of 
rms deviation as shown above. The approach data were analyzed only for those approaches that were flown to 
touchdown. 

The percentage of approaches which meet the four criteria are shown, broken down by display configuration in Table III. 
(The variables, ALS and RVR, were not considered to be important influences in these results and are not shown.) The 
influence of DA is also shown in Table III, where, if the criteria window included “DA,” then the analysis used a 
corresponding altitude window down to 200 ft or to 100 ft as appropriate. 


Table II: Quantitative Approach Performance Standards 



Localizer Tracking 

Glideslope Tracking 

Practical Test Standard 
(PTS) 

Reference 18 

<3/4 Full Scale Deflection (i.e., 1.5 
dots), Between Final Approach Fix and 
Decision Height 

<3/4 Full Scale Deflection, (i.e., 1.5 dots) 
Between Final Approach Fix and Decision 
Height 

AC 120-29, 

Appendix 2, Paragraph 6.2.1. 
Reference 21 

< 1/3 (i.e., 2/3 dots) Full Scale 
Deflection from 1000’ Height Above 
Touchdown (HAT) to 200’ HAT 

< 1/2 Full Scale (i.e., 1 dot) Deflection from 
700’ HAT to 200’ HAT 

“ Cat 2, Successful Approach ” 
FAR Part 91, Appendix A, 
Section 3, Subsection e2 

At 100 ft DH, cockpit is within and 
tracking so as to remain within, the 
lateral confines of the runway 
extended. 

Deviation from glideslope after leaving the 
outer marker does not exceed 50% Full 
Scale Deflection (i.e., 1 dot down to 100 ft 
DH) 

Joint Aviation Regulations - 
All Weather Operations 
AMCAW0 231 
Reference 22 

No more than 5% of approaches with 
>1/3 dot between 300 ft and 100 ft 
HAT 

No more than 5% of approaches with >1 
dot between 300 ft and 100 ft HAT 


There were 200 approaches flown to touchdown using the Baseline-HDD, 185 for the SV-HDD, 205 for the Baseline- 
HUD, and 200 for the SV-HUD, of which 145, 135, 147, and 142 approaches, respectively, were flown using a 100 ft 
DA. Note that the data in Table III for the JAR-AWO just used exceedance data (i.e., approaches where >1/3 dot for 
localizer or >1 dot for glideslope were not found, between 300 ft and 100 ft AFL). Statistical analysis to ensure a 95% 
bound as per JAR-AWO have not yet been performed. Therefore, these results may be considered optimistic against this 
criterion. 


Table III: Percentage Approaches Successfully Meeting Approach Performance Standards 



Localizer 

Glideslope 

HDD 

HUD 

HDD 

HUD 

PTS 200 ft DA Baseline 

100% 

98% 

95% 

98% 

SV 

100% 

100% 

90% 

98% 

1 00 ft DA Baseline 

100% 

100% 

72% 

90% 

SV 

100% 

100% 

73% 

93% 

AC 120-29 200 & 100 Baseline 

100% 

99% 

83% 

95% 

SV 

100% 

100% 

85% 

97% 

FAR 91 100 ft DA Baseline 

100% 

100% 

50% 

79% 




SV 

100% 

100% 

62% 

82% 

JAR-AWO 200 ft DA Baseline 

100% 

100% 

44% 

69% 

SV 

98% 

100% 

42% 

67% 

1 00 ft DA Baseline 

99% 

99% 

50% 

79% 

SV 

99% 

99% 

62% 

82% 


The data of Table III shows that localizer tracking is never a problem, irrespective of the criteria. At most, only 2 
exceedances were encountered for any of the criteria. Glideslope tracking, on the other hand, was a much more stringent 
discriminator and revealed several characteristics which must be considered in using these criteria as quantitative criteria 
for approach and landing operations: 

• Glideslope tracking was always superior using the HUD compared to the HDD. This result is likely due to the 
conformal nature of the HUD (the HDD was a minified display) and its head-up location, as discussed in the 
following. 

• The presence or absence of SV doesn’t appear to dramatically influence the success or failure for glideslope 
tracking although statistical significance has not yet been determined. For the HDD concept, the presence of 
SV does seem to improve acceptable performance rate for approaches going to 100 ft DA (i.e., see FAR-91, 
comparing SV/Baseline above). The presence of SV may be improving the pilot’s ability to interpret the 
guidance in the regime where the glideslope sensitivity is increasing dramatically. Otherwise, SV should not be 
a significant influence; the more important determinants for glidepath tracking should be the quality of the 
guidance and the pilot’s capability and ability to track this information. 

• Down to 200 ft HAT, the pilots could maintain glidepath within 1.5 dots between 90-95% of the time using the 
HDD and 98% with the HUD (i.e., the PTS Standard). When the tighter AC120-29 standard of 1.0 dot is used, 
there is only a minimal drop in HUD performance acceptability (down to 96%) but acceptable HDD 
performance is down to 84%. This performance drop for the HDD condition is likely the result of the pilot 
going “head-out” during this critical time to search for the required visual references while flying head-down 
instruments (i.e., this was a simulated single pilot operation). 

• This “head-out” performance degradation is clearly demonstrated in the JAR-AWO criteria as well. The JAR- 
AWO criteria is the same 1 dot glideslope exceedance, but the altitude range goes down to 100 ft HAT. While 
statistical significance has not yet been computed, the comparison of the 200 ft DA to the 1 00 ft DA data above 
shows how performance changes when the pilots were “visual” (i.e., operations below 200 ft HAT for the 200 ft 
DA condition required that the pilot have identified at least the ALS). The performance changed for the HDD 
and the HUD display conditions. The pilots were likely attending to the visual acquisition task and were not 
focused on tracking the glideslope. 

4.5. Objective Landing Standards Application 

Existing performance-based landing standards were applied in the objective data analysis. These standards were drawn 
from numerous sources pertaining to the general concept of low- visibility approach and landings. However, none of 
these were written specifically as quantitative performance standards for Synthetic and Enhanced System operations. 
The analyses that follow examine their applicability for this purpose. 

In Table IV, the selected quantitative landing performance standards are shown for longitudinal and lateral position at 
touchdown and sink rate at touchdown. Bank angle is typically included in the landing performance standards but was 
excluded from analysis in this paper since no operationally significant differences were found among the main factors as 
described in Section 4.3.2. In addition, many of these standards apply to a Monte Carlo analysis of automatic landing 
system performance and were not necessarily designed to apply to the analyses shown here (i.e., compliance for manual 
flight performance). 

This experiment used a 1000 ft from threshold aim point. For the simulated 757 aircraft, the outboard landing gear 
would be 70 ft from the centerline when the fuselage (the recorded lateral landing position reported herein) is at 58 ft 
lateral deviation from centerline, assuming no crab angle at touchdown. 



Table IV: Quantitative Landing Performance Standards 



Landing Performance 


Longitudinal Position 

Lateral Position 

Touchdown Sink Rate 

Automatic 
Landing 
Systems 
(AC20 - 57A) 
Reference 23 

Two-sigma longitudinal dispersion 
should not exceed 1500 ft total, but 
need not be symmetrical about the 
nominal point. By analysis, 
longitudinal touchdowns should be 
shown to be improbable (10" 6 ) 
outside of a point at least 200 feet 
beyond the threshold and that point 
down the runway at which the pilot 
is in a position to see at least 4 bars 
(on 100’ centers) of the 3000 foot 
touchdown zone lights. 

Two-sigma lateral dispersion of 
aircraft centerline at the main 
gear should not exceed 27 ft 
either side of centerline By 
analysis, lateral touchdowns 
should be shown to be 
improbable (10" 6 ) with the 
outboard landing gear no closer 
than five feet from the lateral 
limits of a 150 foot runway 

No guidance specified. 

AC 120-28D, 
Appendix 3, 
Paragraph 6.3.1. 
Reference 19 

No longitudinal touch down earlier 
than a point on the runway 200 ft. 
(60m) or beyond 2700 ft.(823m) 
from threshold to a probability of 1 

X 10' 6 . 

No lateral touch down with the 
outboard landing gear more than 
70 ft. (21.3m) from runway 
centerline to a probability of 1 x 
10' 6 . 

No sink rate greater than 
structural limit load to a 
probability of 1 x 10" 6 . 

One component of showing 
structural load compliance is 
that the sink rate at touch 
down is not greater than 1 0 
feet per second. 

JAR- A WO 
AMC AWO 231 
Reference 22 

No longitudinal touch down earlier 
than a point on the runway 200 ft. 
(60m) or beyond 2700 ft.(823m) 
from threshold to a probability of 1 
X 10" 6 . 

No lateral touch down with the 
outboard landing gear more than 
70 ft. (21.3m) from runway 
centerline to a probability of 1 x 
10 6 . 

No sink rate greater than 
structural limit load to a 
probability of 1 x 10" 6 . 


4.5.1. Sink rate at Touchdown 

The main factor of RVR (F(2, 97)=3.97, p=0.022) and the second order interaction between DA and location (F(l, 
591)=5.68, p=0.017) were significant for sink rate at touchdown. The other main factors of content, location, ALS, and 
DA and 2 nd order interactions were not significant (p>0.05). Post-hoc tests (SNK using a=0.05) showed two overlapping 
subsets for sink rate at touchdown with the 3 RVR values: 1) 2400 ft (mean= -4.8 fps) and 1800 ft (mean= -5.1 fps) and 
2) 1800 ft (mean= -5.1 fps) and 1200 ft (mean= -5.4 fps). Visual inspection of the data revealed that 29 of the 790 runs 
(or 3.7% of the runs) were not within the acceptable sink rate at touchdown criteria (-10 fps) and that 15 of these 29 
“unacceptable” runs were flown by the one pilot. The other 14 “unacceptable” runs were spread among nine pilots. 

Even though the pilots were not necessarily current in the B-757 aircraft to serve as evaluation pilots, 96% of the data 
were within acceptable sink rate limits. Visibility (i.e., RVR) provided the biggest influence (statistical significance) in 
sink rate at touchdown, but operationally, the differences were marginal. 

4.5.1. Centerline Maintenance 

The main factors of RVR (F(2, 77)=23.24, p<0.001), ALS (F(2, 50)=81.54, p<0.001), and DA (F(l, 23)=72.73, p<0.001) 
and the second order interaction between DA and location (F(2, 591)=37.65, p<0.001) were highly significant for lateral 
distance from centerline (measured in feet). The other main factors (content and location) and the 2 nd order interactions 
were not significant (p>0.05) for this measure. Post-hoc tests (SNK using a=0.05) showed two unique subsets for lateral 
distance from centerline with the 3 RVR values: 1) 1800 ft (mean=4 ft left of centerline) and 2) 2400 ft (mean=8 ft right 
of centerline) and 1200 ft (mean=9 ft right of centerline). There were two unique subsets for ALS: 1) MALSR (mean=l 
ft right of centerline) and 2) ALSF-2 and VFR (both means=13 ft right of centerline). In addition, pilots landed closer to 
the centerline with the 100 ft DA (mean=5 ft right of centerline) than with the 200 ft DA (mean=8 ft right of centerline). 




4.5.2. 


Touchdown Zone Position 


The touchdown aim point position was located 1000 feet from the runway 18R threshold. There were no significant 
(p>0.05) effects for the main factors (content, location, RVR, ALS, and DA) or their interactions in the touchdown 
positions (n=790). The overall mean position was 639 ft forward of the aim point with a standard deviation of 582 ft. 

To be considered a successful landing, the touchdown position had to be between -800 ft aft of the aim point and +1700 
ft forward of the aim point. Visual inspection of the data revealed that 36 out of the 780 runs (or 4.6 % of the runs) were 
not within the acceptable landing criteria as they were greater than 1700 ft forward of the aim point and that 18 of these 
“unacceptable” runs were completed by two pilots (9 runs/pilot). The remaining 18 runs were distributed among 7 
pilots. 75% (27 out of 36 runs) of the unsuccessful landings were made with the 1200 RVR visibility level. 

4.5.3. Landing Performance Discussion 

100 % of the normal runs that ended with a landing were within the centerline criteria and 96% were within the sink rate 
at touchdown criteria for a successful landing as specified in AC 1 20-2 8D/ Appendix 3 19 - namely, they were laterally 
within +/- 58 feet of centerline and with a vertical speed at touchdown of less than 10 fps. RVR, ALS and DA were 
statistically significant for centerline maintenance, but operationally these differences (on the order of one to five feet) 
were negligible. The presence or location of SV imagery didn’t have a significant effect on the pilot’s ability to land on 
centerline. 

Statistical analysis against the criteria using Table IV have not been completed, but cursory inspection of the data 
suggests that the criteria associated with lxl 0" 6 probability requirements will not be met. In Figure 7, the touchdown 
data are shown, broken out by RVR. Included on this plot is the +/-58 lateral and 200 ft to 2700 ft longitudinal 
touchdown zone definition used in Table IV. 
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Figure 7. Landing Performance Data for 1200 RVR (top), 1800 RVR (middle), and 2400 RVR (bottom). 


The data in Figure 7 shows the much higher propensity (but not statistically significant) to land within the “desired” 
touchdown zone with a 2400 RVR compared to the 1800 and 1200 ft RVR visibility conditions. Since this phase of 
flight is strictly “visual” (based on the visibility and the operating criteria used in the test) no influence for SV was 


expected, or seen. In addition, no influence in HUD vs. HDD was seen. Somewhat surprisingly, no operational 
influence in ALS was found. The VFR lighting condition performance was comparable to the ALSF-2 lighting 
condition performance. 


5. CONCLUSIONS 

An experiment was conducted to evaluate SVS and SVS-related technologies to develop operational concepts for all- 
weather approach and landing and to provide quantitative and qualitative information that could be used to develop 
criteria for all-weather approach and landing technologies such as synthetic vision. A Global Positioning System / Wide 
Area Augmentation System (GPS/WAAS) approach was simulated to provide precise approach guidance instead of an 
instrument landing system (ILS), and the “operational considerations” evaluated under this effort included reduced 
visibility, decision altitudes, and airport equipage requirements, such as approach lighting systems, for SVS-equipped 
and non-equipped aircraft. In addition, the role of display location (HDD versus HUD) in landing operations and the 
pilot decision-making process was also investigated. 

Objective results from the present study showed that better approach performance was attainable with the HUD 
compared to the HDD. A slight performance improvement in HDD performance was shown when SV was added, as the 
pilots descended below 200 ft to a 100 ft DA, but this performance was not tested for statistical significance (nor was it 
expected to be statistically significant). The data also showed that precise glideslope tracking was lost once pilots went 
“visual” for the landing. 

Further, the touchdown data showed that regardless of the display concept flown (SV HUD, Baseline HUD, SV HDD, 
Baseline HDD) a majority of the runs were within the performance-based defined approach and landing criteria in all the 
visibility levels, approach lighting systems, and decision altitudes tested. However, a small percentage (5%) of runs 
were flown past the touchdown zone or with a harder landing than specified in the landing criteria. All runs ended safely 
in a full stop with adequate runway remaining. For this visual flight maneuver, RVR appeared to be the most significant 
influence in touchdown performance. The approach lighting system clearly impacted the pilot’s ability to descend to 
100 ft HAT based on existing FAR 91.175 using a 200 ft decision height, but did not appear to influence touchdown 
performance or approach path maintenance. 
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